Fixed and Random Effects Models

How should we pool effect sizes in meta-analysis?

Jason Moggridge

2024-07-30

About Me

  • M. Bioinformatics, U of Guelph
  • B.Sc. Biochem, Carleton
  • Day to day: data analysis, software, and web dev
  • Interests: bio, comp-sci, stats & ML
  • Outdoors: kayak, hike, dive, bike…

Harbour seal (Phoca vitulina) at Ruckle park.

Meta-analysis Recap

Meta-analysis Recap

  • Goal: Estimate the true effect size from multiple studies
  • Steps:
    • define research question (PICOT)
    • collect results from appropriate studies
    • apply statistical model*

Meta-analysis Data

  • dataset is \(K\) studies
  • each independent study is an observation with variables:
    • estimated effect size (\(\widehat{\theta}_{k}\))
    • standard error (\(\epsilon_{k}\) or \(SE_{k}\))
    • sample size (\(n_{k}\))

How should we pool effect sizes?


To get our meta-analysis result, we need to take a weighted average of effect sizes


But how should we weigh the evidence of each study?

Meta-analysis Models

  • Fixed effects and random effects models make different assumptions about how data were generated
  • Model should approximate the underlying processes

Fixed Effects Model

Fixed Effects Assumption

  • One true effect size (\(\theta_{\ F }\))
  • Deviation from \(\theta_{\ F}\) is due to sampling error (\(\epsilon_{\ k}\))

Inverse-Variance Weighting


  • Error (\(\epsilon_{\ k }\)) is inversely proportional to sample size \(N_{k}\).
  • Estimates (\(\widehat{\theta}_{k}\)) with small \(\epsilon_{\ k }\) should be closer to true \(\theta\)
  • These \(\widehat{\theta}_{k}\) should be given more weight for estimating \(\theta\)

Inverse-Variance Weighting


Study weight \(w_{k}\) is inversely related to the variance (\(SE^2_{k}\))

\[ w_{k}=\frac{1}{(SE)^{2}_{k}} \]

The pooled estimate \(\widehat{\theta}\) is the weighted average:

\[ \widehat{\theta} = \frac{\Sigma\ \widehat{\theta}_{k} w_{k}}{\Sigma\ w_{k}} \]

FEM Standard Error

To compute the standard error on our averaged result \(\widehat\theta\)

\[ SE(\widehat{\theta}) = \frac{1}{\Sigma\ w} \]

FEM Assumptions & Interpretation

  • All studies are sampling a completely homogeneous population
  • All studies performed in exactly the same manner.
  • No between-study heterogeneity

. . .

Too simplistic to account for differences between studies.

Random Effects Model

Random Effects Assumption

  • A universe of possible studies
  • Each study has own true effect size (\(\theta_k\))
  • \(\theta_{k}\) are randomly drawn from a distribution with mean true effect size \(\mu\) and variance (\(\tau^2\))

Random Effects Model

The true effect size of study \(k\) (\(\theta_{k}\)) is drawn from a distribution with mean \(\mu\) with error \(\zeta_{k}\).

\[ {\theta}_{k}=\mu+\zeta_{k} \]

The reported result (\(\widehat{\theta}_{k}\)) of a study is generated by the true effect size (\(\theta_{k}\)) plus sampling error (\(\epsilon_{k}\)).

\[ \widehat{\theta}_{k}=\theta_{k} +\epsilon_{k} \]

Random Effects Model

  • We can conceptualize the REM having a hierarchical structure

REM weighting

  • uses the between-study heterogeneity of true effects, \(\tau^2\)
  • \(\tau^2\) increases the relative weight given to smaller studies

\[ w^{*}_{k}=\frac{1}{SE^{2}_{k} + \tau^2} \]

Then compute the weighted average in the usual fashion.

Estimating \(\tau^2\)

  • Beyond the scope of this talk!
  • Many procedures exist (16+ different estimators)
  • DerSimonian and Laird method treated as default, controversially

Exchangeability Assumption

  • \(\zeta_{k}\) is independent of \(k\)
  • there is no way to know what \(\zeta_{k}\) is a priori
  • This assumption may be violated by hidden information, e.g., publication bias

Should we just always use the REM?


  • Only choose the FEM if you have clear reason to (rare)
  • For most scenarios, use the REM.

A Test for Between-Study Heterogeneity


  • does the confidence interval for \(\tau^2\) include zero or not?
  • if not, using REM is justified

References & Further Reading

Questions?